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Understanding  Patterns  of  Team  Collaboration  Employed 

To  Solve  Unique  Problems 

Susan  G.  Hutchins,  Tony  Kendall,  and  Alex  Bordetsky 

Abstract 

‘Macrocognition’  is  a  nascent  area  of  knowledge  engineering  that  focuses  on 
understanding  how  cognition  emerges  in  natural  environments.  One  goal  for 
studying  macrocognition  is  to  understand  the  complexity  entailed  in  inter-  and 
intra-individual  cognition.  The  goal  of  the  research  reported  here  is  to  better 
understand  how  team  collaboration  influences  and  facilitates  the  team’s  task 
performance.  In  this  paper  we  describe  our  analysis  of  several  complex  team 
collaboration  tasks:  (a)  firefighters  from  the  Fire  Department  of  New  York  on 
September  11,  2001,  (b)  air  warfare  teams  on  an  Aegis  ship,  and  (c)  the  team 
collaboration  entailed  in  conducting  Maritime  Interdiction  Operations.  Team 
communications  that  transpired  during  three  complex  problem  solving  situations 
were  analyzed  to  understand  how  teams  collaborate  to  create  new  knowledge  and 
decide  on  a  course  of  action  during  complex,  one-of-a-kind  problems.  Commun¬ 
ications  were  analyzed  using  definitions  of  cognitive  processes  included  in  a 
conceptual  model  of  team  collaboration.  These  processes  include:  (1)  individual 
knowledge  building,  (2)  developing  knowledge  interoperability,  (3)  developing 
team  shared  understanding,  and  (4)  team  consensus.  The  way  the  team’s  cognitive 
behavior  maps  to  the  model  of  team  collaboration  is  discussed  along  with 
differences  in  patterns  of  collaboration  for  different  decision-making  domains. 

INTRODUCTION 

‘Macrocognition’  is  an  emerging  field  within  the  area  of  cognitive  engineering  that  describes  the 
way  cognition  occurs  in  naturalistic,  or  real-world,  decision-making  events.  Macrocognition  is 
defined  as  the  internalized  and  externalized  high-level  mental  processes  employed  by  teams  to 
create  new  knowledge  during  complex,  one-of-a  kind,  collaborative  problem  solving  (Letsky, 
Warner,  Fiore,  Rosen,  and  Salas,  2007).  The  goal  for  this  research  is  to  understand  the  role  of 
cognition  in  teams  who  are  collaborating  to  solve  complex,  one-of-a  kind  problems,  such  as:  (a) 
firefighters  responding  to  the  events  of  September  11,  2001,  (b)  a  boarding  party  conducting 
Maritime  Interdiction  Operations  (MIO),  (c)  a  Navy  shipboard  combat  information  center  team 
responding  to  air  warfare  threats,  or  (d)  an  ad  hoc  inter-agency  team  responding  to  a 
humanitarian  assistance/  disaster  relief  situation.  This  research  focuses  on  the  contextually  bound 
processes  entailed  in  sense  making,  managing  uncertainty,  and  related  cognitive  processes 
entailed  in  responding  to  emerging  events  that  occur  in  dynamic  decision-making  situations 
(Fiore,  Rosen,  Salas,  Burke,  &  Jentsch,  2008). 

Team  communications  data  from  three  decision-making  situations  (a  series  of  field  experiments, 
a  series  of  laboratory  experiments,  and  a  real-real  event)  where  a  team  of  people  collaborate  to 
solve  a  complex  problem  were  analyzed  and  reported  on  previously  (Hutchins,  Bordetsky, 
Kendall,  and  Garrity,  2007).  This  paper  extends  our  analysis  by  examining  similarities  and 
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differences  in  team  collaboration  patterns  across  three  decision-making  domains.  In  this  series  of 
studies  we  analyzed  data  obtained  from  teams  and  tasks  characterized  by  the  following 
descriptions.  Team  types  include  teams  who  employ  asynchronous  communications  among 
distributed  team  members,  may  be  culturally  diverse,  and  where  team  members  bring  their 
heterogeneous  knowledge  to  bear  to  solve  the  problem.  In  these  complex  situations  each  team 
member  plays  a  unique  role,  team  members  operate  within  a  command  structure,  and  in  some 
situations  involve  rotating  team  members.  Collaborative  problem-solving  situations  include  time 
pressure,  information/  knowledge  uncertainty,  dynamic  information,  large  amounts  of  knowledge 
(cognitive  overload),  and  human-agent  interface  complexity. 

The  purpose  of  this  effort  is  to  provide  support  to  the  Office  of  Naval  Research  (ONR) 
Collaboration  and  Knowledge  Integration  (CKI)  Program’s  goal  to  improve  the  effectiveness  of 
team  decision  making  in  complex,  data-rich  situations.  This  includes  developing  a  better 
understanding  of  the  cognitive  processes  employed  when  teams  collaborate  to  solve  problems. 
The  focus  is  on  FORCEnet  type  issues  such  as  the  MIO  task  that  entails  detecting  a  moving 
vessel  emitting  signs  of  ionizing  radiation,  followed  by  a  boarding  of  that  vessel.  The  vessel  is 
boarded  to  take  fingerprints  or  other  biometrics  and  to  perform  a  search  for  contraband.  Results 
of  the  boarding  may  need  to  be  communicated  back  to  command  elements  on  land  and  to  remote 
experts  for  analysis.  After  successful  analysis,  key  decision  makers  may  collaborate  to  determine 
the  appropriate  course  of  action. 

Team  Collaboration 

Collaborating  teams  are  ubiquitous  with  examples  of  tasks  requiring  collaboration  encompassing 
an  ever  increasing  range  of  situations.  Medical  specialists  in  India  provide  second  opinions  and 
interpret  medical  results  at  times  when  medical  specialists  in  the  U.S.  are  in  short  supply  (e.g., 
during  weekends  or  in  the  middle  of  the  night)  (Friedman,  2006)  and  accountants  half-way 
around  the  globe  prepare  income  taxes  at  a  fraction  of  the  cost  compared  with  U.S.  accountants 
(Friedman,  2006).  Scientists  collaborate  with  researchers  on  a  remote  space  station;  and  people 
on  land  collaborate  with  robotic  geologists  (Clancy,  2004).  Military  teams  collaborate  on  plans 
and  execution  for  a  wide  variety  of  mission  areas.  These  teams  all  have  characteristics  in 
common  -  they  engage  in  tasks  that  involve  critical  decisions  and  must  often  coordinate  their 
activities  to  accomplish  these  tasks  effectively. 

Model  of  Team  Collaboration 

In  this  paper  we  report  on  research  conducted  to  assist  in  evaluating  and  refining  the  ONR  CKI 
Program’s  model  of  team  collaboration  developed  by  Warner,  Letsky,  and  Cowen  (2005).  This 
conceptual  model  emphasizes  the  cognitive  aspects  of  team  collaboration  and  includes  the  major 
human  decision-making  processes  used  during  team  collaboration.  Operational  tasks  performed 
by  teams  whose  collaborative  cognitive  behavior  was  analyzed  include  team  data  processing, 
team  decision  making,  course  of  action  selection,  and  developing  shared  understanding.  The  goal 
for  this  research  is  to:  (a)  gain  insight  into  where  the  model  can  be  improved  and  (b)  detennine 
how  the  macrocognitive  processes  included  in  the  model  contribute  to  team  performance.  The 
focus  of  the  collaboration  model  is  on  knowledge  building  among  the  team  members  and 
developing  team  consensus  for  selection  of  a  course  of  action. 
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Problem  Area 
Characteristics 


Collaborative  Situation 


Collaboration  Stages  &  Cognitive  Processes 


•  time  pressure 

•  information/knowledge 
uncertainty 

•  dynamic  information 

•  large  amount  of  knowledge 
(cognitive  overload) 

•  human-agent  interface 


•  asynchronous 

•  distributed 

•  culturally  diverse 

•  heterogeneous  knowledge 

•  unique  roles 

•  command  structure 
(hierarchical  vs.  flat) 

•  rotating  team  members 

Operational  Tasks 


•  team  decision  making,  COA 
selection 

•  develop  shared  understandir 

•  intelligence  analysis 
(team  data  processing) 


|  Meta-Cognitive: 

J  •  individual  conversion  of 
i  data  to  knowledge 

f  — — - 

Maero-Cognitive: 


•  individual  mental  model 
construction 

•  knowledge  interoperability 
development 

•  individual  task,  team  and  domain 
knowledge  development 

•  individual  knowledge  object 
development 

•  individual  visualization  and 
representation  of  meaning 


•  team  integration  of  individual  j  •  team  agreement  on  a  commoi 
knowledge  for  common  understanding  solution 


•  knowledge  interoperability 
development 

•  iterative  information  collection 

and  analysis 

•  team  shared  understanding 
development 

•  develop,  rationalize,  &  visualize 
solution  alternatives 

•  convergence  of  individual  mental 
models  to  team  mental  model 

•  individual  task,  team  and  domain 
knowledge  development 


•  team  negotiation  of  solution 
alternatives 

•  team  pattern  recognition 

•  team  shared  understanding 
development 

•  convergence  of  individual 
mental  models 

•  critical  thinking 

•  sharing  hidden  knowledge 

•  individual  task  knowledge 
development 

•  team  task  knowledge 
development 


•  solution  adjustment  to 
fit  goals  and  exit  criteria 


•  compare  problem  solution 
against  goals 

•  team  shared  understanding 
development 

•  convergence  of  individual 
mental  models  of  solutior  i 

•  analyze,  revise  output 


i 


Mechanisms  for  achieving  Meta  and  MacroCogiritive  Processes  (applies  to  all  stages) 

•  Verbal  communications:  representing  and  discussing  individual  information,  discussing  team  generated  information, 
questioning,  agreeing  /  disagreeing,  negotiating  perspectiyes,  discussing  possible  solutions,  providing  rationale. 

*  Non-Verbal  communications:  facial  expressions,  voice  clues  (vocal  paralanguage),  hand  gestures,  body  movements(kinesics) 


Figure  1.  Model  of  Team  Collaboration.  (From  Warner,  Letsky,  &  Cowen,  2004). 


Taking  Actions  as  Part  of  Problem  Solving 

Many  tasks  require  rapid  and  accurate  coordination  of  information  and  behavior  to  successfully 
cope  with  the  demands  of  time-compressed,  ambiguous  situations  with  the  threat  of  fatal 
consequences.  The  types  of  tasks  that  are  described  by  the  model  of  team  collaboration  are 
naturalistic  decision  making  tasks,  that  is,  where  the  focus  is  on  recognition  of  situations  and 
reflection  processes  where  there  is  a  continuous  shift  between  thought  and  action  (Lipshitz  and 
Ben  Shaul,  1996;  Espevik,  Johnson,  Eid,  and  Thayer,  2006).  In  naturalistic  decision-making 
tasks,  where  a  recognition-primed  decision  strategy  is  typically  employed,  the  current  situation  is 
compared  with  previously  encountered  situations,  and  actions  that  were  taken  in  past  similar 
situations  that  were  successful  in  those  situations,  and  the  associated  outcomes  that  were 
produced  are  used  to  make  rapid  decisions.  Decision  makers  use  their  previous  experience  to 
implement  different  solutions  to  a  series  of  problems  (Espevik  et  al,  2006). 

Orasanu  (1994)  observes  that  team  communications  are  used  to  accomplish  tasks  and  also 
indicate  the  cognitive  processes  of  individual  team  members.  Following  Orasanu,  team 
communications  serve  three  main  functions  related  to  problem  solving:  sharing  information, 
directing  actions,  and  reflecting  thoughts.  Sharing  information,  in  the  three  decision-making 
domains  covered  in  this  paper,  is  used  to  inform  the  other  team  members  of  the  evolving  status 
of  the  situation  regarding  a  team  member’s  understanding  of  the  situation  and  to  update  team 
members  as  the  team  takes  various  actions  to  deal  with  the  problem.  Depending  on  the  decision¬ 
making  domain,  much  of  this  communication  relies  on  standard  operating  procedures  to  provide 
standardized  ways  of  communicating,  types  of  information  to  share,  and  pre-defined  actions  to 
be  taken  in  response  to  specific  situations. 
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Directing  action  is  the  second  critical  function  of  team  communications.  This  includes  direct 
commands  for  routine  actions  for  a  wide  variety  of  situations.  It  also  includes  action  commands 
for  dealing  with  problems,  such  as  gathering  information  for  a  decision.  In  line  with  Orasanu’s 
analysis,  the  third  function  of  team  communications  is  sharing  what  one  is  thinking,  as  opposed 
to  passing  on  information  obtained  from  an  external  source.  This  type  of  communication  is 
essential  to  developing  a  shared  problem  model  among  team  members. 

METHOD 

Verbatim  transcripts  were  analyzed  from  two  series  of  experiments,  and  one  real-world  event 
where  teams  collaborated  to  solve  a  complex  problem:  (a)  MIO,  (b)  air  warfare  decision  making, 
and  (c)  the  communications  that  transpired  between  the  firefighters  of  the  Fire  Department  of 
New  York  (FDNY),  on  Sept.  11,  2001  (Fire  Department  of  the  City  of  New  York,  2001). 
Transcripts  included  communications  that  occurred  between  all  team  members  as  well  as  with 
decision  makers  at  the  distributed  sites.  Team  communications  data  were  analyzed  and  coded 
using  the  cognitive  process  definitions  developed  by  Warner,  et  ah,  (2004).  This  research  builds 
on  previous  work  to  validate  this  model  (Warner,  et  al,  2004).  The  current  effort  uses  a  similar 
methodology  applied  to  three  different  decisiomnaking  scenarios.  For  definitions  of  the  macro- 
cognitive  processes  include  in  the  model  see  Hutchins,  Bordetsky,  Kendall,  and  Garrity,  2007. 

Team  Collaboration  Tasks 

In  all  three  problem-solving  tasks  decision  making  is  difficult  because  the  available  information 
is  often  incomplete  or  ambiguous.  Team  members  possess  highly  specialized  knowledge  that 
must  be  synthesized  for  all  team  members  to  develop  a  shared  understanding  of  the  problem:  The 
inherent  time  pressure  in  these  types  of  tactical  situations  creates  a  challenging  decision-making 
situation.  Problem  solving  situations  that  make  use  of  multiple  information  sources  and  require 
collaborative  analysis  to  develop  a  team  shared  understanding  of  the  task,  the  environment,  and 
the  status  of  other  collaborators,  enhance  the  richness  of  the  communications.  A  key  enabler  of 
collaborative  decision  making  is  a  virtual  aggregation  of  individuals,  organizations,  systems, 
infrastructure,  and  processes  to  create  and  share  data,  infonnation,  and  knowledge  that  is  needed 
for  the  team  to  plan,  execute,  and  assess  actions/operations.  This  team  collaboration  should 
enable  a  leader  to  make  decisions  better  and  faster  than  were  previously  possible. 

Collaborative  information  environments  are  intended  to  transform  collaborative  planning  and 
decision  making  from  a  relatively  sequential,  hierarchical  process  to  a  more  parallel  approach 
that  allows  ‘virtual’  collaborative  interaction  by  all  organizations  regardless  of  their  location. 
Important  components  of  collaborative  environments  include  collaborative  decision  support  and 
situational  awareness  tools  to  support  the  overall  task. 

Cognitive  Complexity  of  Scenarios 

Brief  descriptions  of  the  decision-making  domains  analyzed  for  this  effort  are  provided  below. 

Maritime  Interdiction  Operations.  Scenarios  used  for  the  MIO  experiments  focus  on  detecting, 
identifying,  and  interdicting  nuclear  materials  in  open  waters.  The  critical  task  involves  the 
cognitively  complex  issue  of  discrimination,  that  is,  how  to  detennine  the  presence  of  contraband 
radiological  material  against  a  background  containing  multiple  benign  radiation  sources.  A 
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variety  of  benign  sources,  such  as  smoke  detectors,  radiant  signs,  mantles  used  for  camping 
lanterns,  and  a  container  load  of  bananas  all  can  cause  radiation  detectors  to  alarm.  For  example, 
“smoke  detectors  contain  small  amounts  of  americium,  radiant  signs  glow  because  they  contain 
tritium,  a  radioactive  hydrogen  isotope,  and  bananas,  contain  a  small  fraction  of  potassium-40 
which  emits  ionizing  radiation”  (Schwoegler,  2006,  p.  6). 

Technical  expertise,  provided  by  remotely-located  experts,  is  required  to  interpret  the  radiolog¬ 
ical  signals  emitted  from  complex  detectors  to  enable  on-site  personnel  to  make  the  fine 
discriminations  required.  The  boarding  party  is  comprised  of  Coast  Guard  officers,  as  well  as 
members  of  other  local  law  enforcement  agencies  who  collaborate  with  remotely-located  experts. 
During  a  MIO,  distributed  experts  collaborate  to  detennine  whether  the  detected  material 
represents  a  risk  and  support  the  on-site  operators  in  deciding  how  to  handle  suspect  material. 
Performing  these  complex  discriminations  is  made  possible  by  the  collaborative  capability 
provided  by  the  collaborative  workspace  in  terms  of  bringing  remote  expertise  to  the  vessel 
undergoing  the  search  and  the  ability  to  rapidly  send  and  receive  communications  and  data  files 
between  a  diverse  team  of  experts  who  all  bring  their  respective  expertise  to  bear  to  deal  with  a 
potentially  high-threat  situation. 

The  objective  of  the  MIO  experiments  is  to  evaluate  the  use  of  networks,  advanced  sensors,  and 
collaborative  technology  for  conducting  rapid  MIOs.  Specifically,  the  ability  of  a  boarding  party 
to  rapidly  set  up  ship-to-ship  and  ship-to-shore  communications  in  order  to  simultaneously 
search  for  radiation/  explosive  sources  and  conduct  biometrics  identification  while  maintaining 
contact  with  the  boarding  vessel  and  remote  collaborators.  The  goal  is  to  enable  geographically 
distributed  command  centers  and  subject  matter  experts  to  collaborate  with  the  boarding  party  in 
real  time,  facilitating  situational  understanding,  course  of  action  selection,  and  execution 
(Bordetsky,  Clement,  and  Vega,  2007). 

Air  Warfare.  Air  warfare  decisionmaking  is  conducted  in  the  combat  information  center  (CIC) 
of  a  Navy  ship.  The  team  is  responsible  for  identifying  and  responding  to  a  large  number  of  air 
tracks  under  high  time  pressure.  These  air  tracks  can  fit  multiple  hypotheses  regarding  the  level 
of  threat  they  pose  to  the  battlegroup  due  to  the  high  level  of  ambiguity  associated  with  the  data. 
Incoming  information  arrives  via  various  sensor  systems  (radar,  electronic  support  measures 
system,  identification  friend  or  foe,  etc.),  and  various  reports,  e.g.,  intelligence  reports,  passed 
by  other  platfonns  in  the  area.  Communications  between  team  members  are  passed  as  soon  as 
information  is  received  and  updated  reports  are  passed  as  soon  as  new  information  is  obtained 
for  any  track.  In  a  series  of  speech  turns,  five  separate  contacts  may  be  discussed  at  various 
levels  -  initial  reports,  updated  reports,  sharing  information  on  the  response,  or  lack  of  response, 
by  the  contact  to  some  action  taken  by  the  ship,  etc. 

Six  collocated  team  members  communicated  with  several  non-collocated  information  sources, 
e.g.,  the  battle  group  commander,  the  Saudi  air  tower,  assets  passing  intelligence  reports,  other 
ships  and  friendly  aircraft  in  the  vicinity  of  the  battlegroup,  to  gather  additional  infonnation 
from  them  and  keep  them  apprised  of  the  unfolding  scenario  as  they  collaborated  to  identify  air 
tracks.  The  global  air  warfare  task  involves  identification  and  responding  to  numerous  contacts. 
When  an  aircraft  is  detected,  CIC  personnel  work  as  a  team  to  detennine  the  identity  and  to  try 
to  determine  if  the  aircraft  poses  a  threat.  The  high  degree  of  ambiguity  associated  with  contact 
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information  can  often  make  threat  assessment  very  difficult  because  many  pieces  of  data  can  tit 
multiple  hypotheses  regarding  threat  assessment  (Hutchins,  Morrison,  &  Kelly,  1996).  The 
global  response  choices  (that  is,  do  nothing,  monitor,  take  various  actions  to  determine  intent,  or 
engage)  are  largely  detennined  by  the  ship’s  orders  and  the  current  geopolitical  situation. 
Specific  actions  depend  on  the  local  conditions  and  the  relative  positions  of  the  inbound  contact 
of  interest  and  own  ship. 


RESULTS 

Table  1  depicts  the  results  of  analysis  of  three  collaboration  tasks:  four  air  warfare  teams,  three 
MIO  teams,  and  the  collaborative  decision-making  that  transpired  between  firefighters  from  the 
FDNY.  The  percentage  of  speech  turns  for  air  warfare,  MIO,  and  the  9-11  firefighting  tasks 
coded  as  representing  each  of  the  macrocognitive  processes  included  in  the  model  is  presented  in 
Table  1.  Analysis  of  the  communications  that  transpired  between  firefighters  and  the  dispatcher 
on  September  1 1  indicates  these  firefighters  used  of  more  of  the  cognitive  processes  included  in 
the  model  compared  to  teams  during  MIO  experiments  and  air  warfare  decision  making. 
(However,  differences  in  the  percentages  of  the  cognitive  processes  used  may  also  reflect  the 
larger  sample  size  of  speech  turns  in  the  9-11  data.  The  number  of  speech  turns  for  each  team  is 
included  in  the  bottom  row  of  Table  1.) 

In  general,  the  large  number  of  speech  turns  coded  as  categories  1-4  of  Table  1  reflects  the  huge 
emphasis  on  individual  knowledge  construction  that  is  required  for  all  three  tasks.  Of  particular 
interest  is  the  large  number  of  speech  turns  coded  as  individual  task  knowledge  (ilk)  development 
which  reflects  the  high  degree  of  uncertainty  inherent  in  all  three  decision-making  tasks.  ( Itk  is 
defined  as  a  team  member  asking  for  clarification  to  data  or  information).  These  high 
percentages  of  use  of  itk  by  decision  makers  in  all  three  task  domains  indicate  these  teams 
devoted  a  large  amount  of  effort  to  trying  to  manage  this  task  uncertainty. 


Table  1.  Percentage  of  Macrocognitive  Processes  Used  by  Teams  for  Three  Different  Tasks. 


Air  Warfare  Scenarios 

MIO  Scenarios 

Sept.  11,2001 

Macrocognitive  Process  Coding 
Categories 

Seen  D 
Run  A 

Seen  D 

Run  B 

CG 

59 

DDG 

54 

Nov 

06 

June 

06 

Sept 

06 

Firefighters' 

9-11 

Knowledge  Construction 

1. 

Data  to  information  (dti) 

02 

04 

- 

22 

03 

13 

- 

<01  (2) 

2. 

Individual  mental  model  (imm) 

14 

11 

14 

15 

01 

18 

07 

02  (14) 

3. 

Individual  task  knowledge  development  (itk) 

43 

29 

24 

17 

52 

18 

40 

41  (325) 

4. 

Team  knowledge  development  (tk) 

19 

05 

14 

01 

04 

13 

07 

27  (210) 

5. 

Knowledge  object  development  (ko) 

- 

- 

- 

- 

- 

05 

07 

- 

6. 

Visualization  and  representation  (vnn) 

- 

- 

- 

- 

- 

- 

- 

- 

Collaborative  Team  Problem  Solving 

7. 

Common  understanding  (cu) 

- 

06 

- 

03 

15 

06 

02  (16) 

8. 

Knowledge  interoperability  (kio) 

- 

05 

- 

01 

03 

- 

08 

01  (8) 

9. 

Iterative  collection  and  analysis  (ica) 

02 

11 

- 

- 

09 

10 

12 

- 

10. 

Team  shared  understanding  (tsu) 

02 

16 

22 

20 

04 

05 

03 

<01  (6) 

11. 

Solution  alternatives  (sa) 

- 

03 

- 

- 

09 

- 

- 

02  (13) 

12. 

Convergence  of  mental  models  (cmm) 

02 

02 

- 

- 

01 

- 

- 

03  (22) 
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13. 

Agreement  on  common  solution  (cs) 

- 

- 

- 

- 

- 

- 

<01  (1) 

Team  Consensus 

14. 

Team  negotiation  (tn) 

- 

- 

- 

- 

06 

- 

- 

<01  (1) 

15. 

Team  pattern  recognition  (tpr) 

- 

- 

- 

- 

- 

- 

- 

<01  (3) 

16. 

Critical  thinking  (ct) 

- 

- 

- 

- 

- 

- 

- 

<01  (3) 

17. 

Sharing  hidden  knowledge  (shk) 

- 

02 

- 

- 

- 

- 

- 

<01  (5) 

18. 

Solution  adjustment  against  goal  (sag) 

- 

- 

- 

- 

- 

- 

- 

Outcome  Evaluation  and  Revision 

19. 

Compare  solution  options  against  goals  (csg) 

- 

01 

- 

- 

- 

- 

- 

<01  (2) 

20. 

Analyze,  revise  solutions  (aro) 

- 

- 

- 

- 

- 

- 

- 

<01  (1) 

21. 

Miscellaneous  (misc) 

40 

21 

30 

26 

08 

- 

- 

52  (849) 

Decision  to  Take  Action 

22. 

Issue  order  regarding  course  of  action  (coa) 

12 

05 

13 

22 

- 

- 

02 

12  (92) 

23. 

Request  take  action  (rta) 

05 

02 

14 

05 

01 

05 

09 

07  (53) 

Total  number  of  speech  turns  for  each  team 

96 

131 

187 

233 

73 

40 

118 

777 

Numbers  in  parentheses  indicate  the  number  of  speech  turns  in  each  macrocognitive  category. 

A  majority  of  the  communications  for  9-11  involved  knowledge  construction  and  significantly 
less  team  communications  for  the  other  stages:  collaborative  team  problem  solving,  team  consen¬ 
sus,  and  outcome  evaluation  and  revision.  Looking  across  all  three  decision-making  domains, 
evidence  was  found  for  all  seven  macrocognitive  processes  that  occur  during  the  collaborative 
team  problem  solving  phase  (categories  7-13  in  Table  1),  where  teams  integrate  individual 
knowledge  to  develop  a  team  common  understanding,  indicating  the  role  these  cognitive 
processes  play  for  teams  who  engaged  in  all  three  tasks.  Far  fewer  speech  turns  were  coded  as 
representing  processes  included  in  the  team  consensus  phase  of  collaboration  (categories  14-18 
in  Table  1).  The  macrocognitive  codes  with  the  smaller  percentages  show  that  the  FDNY  did  not 
reach  the  later  stages  included  in  the  team  collaboration  model.  Most  of  their  efforts  were  aimed 
at  trying  to  figure  out  what  was  going  on  around  them,  which  focused  on  gathering  information 
and  developing  knowledge,  and  creating  a  mental  model  within  which  they  could  work.  The 
small  percentage  of  speech  turns  that  were  coded  as  macro-cognitive  processes  associated  with 
team  consensus  and  outcome  evaluation  and  revision  indicated  that  the  course  of  action  selection 
phase  of  air  warfare,  in  particular,  and  to  a  lesser  degree,  MIO  and  firefighting,  is  not  conducted 
in  a  collaborative  manner. 

Patterns  of  Macrocognitive  Processes  Used  by  Firefighters 

One  striking  difference  between  the  9-11  data  and  the  data  from  the  two  experiments  (air  warfare 
and  MIO)  is  the  difference  in  the  number  of  speech  turns  coded  as  individual  task  knowledge 
(i itk )  development.  Twenty-six  percent  of  the  total  number  of  speech  turns  for  the  combined  air 
warfare  and  MIO  experiments  were  coded  itk  compared  to  a  significantly  larger  percentage,  i.e., 
41%,  for  9-1 1  data  (p<.0001,  z=6.5,  combined  n=1555).  The  second  largest  difference  in  the  use 
of  the  macrocognitive  processes  was,  or  team  knowledge  ( tk )  development,  which  comprised  7% 
for  the  combined  experimental  groups  and  27%  for  the  9-11  data.  Again,  the  difference  was 
significant  (p<.0001,  z=11.03,  n=1555).  Itk  and  tk  combined  represent  32%  of  the  experimental 
teams’  speech  turns  and  a  majority  of  speech  turns  for  the  9-11  team  (69%).  A  majority  of  the 
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communications  for  9-11  involved  knowledge  construction  because  of  the  completely 
unanticipated  nature  of  the  unfolding  terrorist  attack  scenario. 


To  see  the  full  cycle  of  the  collaboration  stages  and  macrocognitive  processes  at  work  in  the 
team  collaboration  for  the  FDNY  on  Sept.  11,  the  larger  problem  facing  firefighters  of  search 
and  rescue,  and  extinguishing  the  fire  in  the  twin  towers  of  the  World  Trade  Center  (WTC)  needs 
to  be  broken  into  four  smaller  problems  that  correspond  with  each  event  that  transpired  that  day. 
These  events  and  the  problems  and  resultant  tasks  to  be  performed  to  deal  with  each  event  are 
listed  in  Table  2.  The  FDNY  team’s  cognitive  processing  of  the  tasks  for  each  of  these  four 
events  follows  the  collaboration  stages  in  the  team  collaboration  model.  For  example,  when  the 
first  plane  hit  the  WTC,  the  problem  involved  determining  what  happened  and  creating  a  mental 
model  within  which  the  FDNY  could  work.  With  each  new  event  (each  aircraft  hitting  the  WTC 
towers,  collapse  of  the  North  and  South  Towers)  the  team  had  to  form  a  new  team  problem 
model. 


Table  2.  Events,  Team  Mental  Model  and  Tasks  for  FDNY  on  Sept  11,  2001. 


Time 

Event 

Team  Mental  Model 

Situation/  Tasks 

8:47- 

9:02 

First  plane  hit  the 

World  Trade  Center 

Between  plane  impacts,  it  still 
could  have  been  an  accident 

•  Evacuate  South  Tower 

•  Gain  knowledge  of  situation 

•  Assign  units  to  respond  to  the  scene 

9:03- 

9:58 

Second  plane  hit  the 
World  Trade  Center 

Definitely  an  attack  but  buildings 
could  still  withstand  the  impacts 

•  Had  to  divide  their  resources  between 
the  two  tower  lobbies  for  command 
posts 

9:59- 

10:28 

South  Tower  collapsed 

Complete  chaos 

•  How  much  was  lost? 

•  How  many  people  were  lost? 

•  Complete  loss  of  SA  (including  C2) 

•  Evacuate  the  North  Tower 

10:29- 

11:07 

North  Tower  collapsed 

Complete  chaos 

•  No  idea  of  the  breadth  of  the  loss, 
shock  and  grief 

•  Rescue  for  any  lost  in  the  nibble 

Firefighters  used  19  of  the  23  macrocognitive  processes,  listed  in  Table  1,  included  in  the  model 
[Note:  Table  1  includes  the  two  new  macrocognitive  codes  that  were  a  product  of  this  research, 
22  and  23].  This  includes  all  codes  except:  knowledge  object  (ko)  development,  which  requires 
pictures  and  icons;  individual  visualization  and  representation  of  meaning  ( vrm ),  which  requires 
visual  aids;  iterative  information  collection  and  analysis  ( ica ),  which  entails  collecting  and 
analyzing  information  without  mentioning  a  solution,  and  solution  adjustment  against  goal 
( saag )  and  exit  criteria,  where  the  team  compares  the  solution  option  against  goal  and  exit 
criteria.  These  four  macrocognitive  processes  did  not  pertain  to  the  FDNY  radio  communications 
for  either  task-  or  standard  operating  procedures  (SOP)-specific  reasons. 

Both  ko  and  vrm  are  not  possible  on  the  radio  (because  the  handie  talkies  do  not  include  a  video 
screen),  but  are  used  in  the  incident  command  center  in  the  lobby  of  the  two  towers  using  the 
board  with  the  location  of  the  fire,  operations  post,  staging  area,  and  responding  units,  and  also  at 
the  dispatcher’s  office  with  the  map  of  the  location  of  all  the  responding  units.  No 
communications  were  coded  as  ica  because  use  of  the  FDNY  highly-refined  SOP  dictates  that 
firefighters  always  comment  and  collaborate  towards  a  solution  with  each  speech  turn  (in  other 
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words,  they  always  noted  what  they  were  doing  about  a  reported  situation,  as  they  reported  on  a 
situation),  due  to  the  inherent  time  crunch.  In  terms  of  use  of  saag,  the  firefighters  did  not  think 
they  needed  to  think  about  their  exit  strategy  as  they  never  got  to  complete  their  tasks. 

Of  the  total  1620  total  speech  turns  for  the  firefighters,  849  (52%)  were  coded  as  miscellaneous, 
and  were  thus  removed  from  further  analysis,  leaving  771  speech  turns.  (Communications  were 
coded  as  misc  when  it  was  simply  an  acknowledgement  of  receipt  of  the  message,  or  an  initial 
attempt  to  get  the  attention  of  the  person  for  whom  the  message  was  intended.  These  misc 
communications  are  important  for  closed-loop  communications,  for  example,  where  it  is 
important  to  let  the  message  sender  know  the  message  was  received,  e.g.,  “roger,”  or  “10-4.”) 

The  most  frequently  used  macrocognitive  processes,  in  order  of  usage  included:  (41%)  iterative 
team  knowledge  development  ( itk ):  indicating  the  team  was  asking  lots  of  questions.  This  raises 
the  issue  of  how  to  alleviate  the  need  for  so  many  questions  and  the  very  high  level  message 
traffic?  (27%)  -  Developing  team  knowledge  (ik):  which  refers  to  sharing  knowledge  with  fellow 
firefighters  and  passing  knowledge  back  to  the  dispatcher;  (12%)  -  Course  of  action  (coa): 
telling  the  dispatcher  and/or  other  responding  units  what  to  do;  (7%)  -  Request  take  action  ( rta ): 
Requesting  something  of  the  dispatcher  or  responding  units;  (3%)  -  Constructing  team  mental 
model  ( cmm ):  (2%)  -  Developing  common  understanding  ( cu );  and  (2%)-  Individual  mental 
model  ( imm ):  Individuals  contributing  to  the  team’s  mental  model. 

Taking  Actions  as  Part  of  Problem  Solving 

Much  of  the  radio  communication  revolves  around  asking  fellow  firefighters  to  respond  or  to  do 
something  towards  the  team’s  goal.  This  is  seen  in  codes  22  and  23  in  Table  1:  course  of  action 
(coa,  12%)  and  request  to  take  action  (rta,  7%).  These  codes  show  the  firefighters  taking  actions 
to  move  toward  accomplishing  their  end  goals,  with  the  knowledge  that  they  do  not  have  to  carry 
out  the  plan  of  action  themselves  with  the  capacity  to  ask  for  support  from  their  fellow 
responders.  These  new  macrocognitive  categories  both  entail  a  decision  by  the  team  member  that 
an  action  needs  to  be  taken. 

Patterns  of  Team  Collaboration 

Since  the  FDNY  team  communications  data  from  Sept  1 1  is  richer  than  what  can  be  represented 
by  numbers  in  a  table,  we  developed  a  way  to  graphically  depict  the  team’s  pattern  of 
collaboration  to  represent  their  traversing  through  the  stages  of  the  model  of  team  collaboration. 
One  way  to  gain  insight  into  a  team’s  cognitive  processes  employed  during  collaboration  is  to 
examine  the  pattern  of  communications,  reflecting  the  macrocognitive  processes  as  they 
communicated  and  responded  to  the  fires  in  the  towers  of  the  WTC.  Following  our  decision  to 
divide  the  2.5  hours  of  team  communications  into  four  phases  corresponding  to  the  four  major 
events  of  9-11,  a  subset  of  the  771  speech  turns  was  selected  to  characterize  and  graphically 
represent  the  team’s  cognitive  processing  of  those  events  in  relation  to  the  macrocognitive 
processes  included  in  the  model.  Criteria  for  selecting  speech  turns  were  when  (a)  a  new  topic 
was  discussed  or  (b)  a  significant  request  was  made  for  a  team  member  to  take  action  or  a  course 
of  action  was  initiated.  One  of  the  authors  read  through  the  transcript  and  noted  each  time  a  new 
topic  or  event  occurred  in  the  transcript  or  a  significant  course  of  action  was  issued.  A  second 
author  reviewed  the  communications  selected  for  inclusion  in  these  figures  and  made  some 
minor  adjustments. 
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Figures  2a  and  2b  depict  this  high-level  representation  of  the  firefighters’  collaborative  pattern  as 
they  responded  to  the  first  event:  When  the  first  plane  hit  the  North  Tower  of  the  WTC.  The  goal 
was  to  show  the  order  in  which  the  firefighters  used  the  macrocognitive  processes  during  the  first 
phase  of  the  Sept.  1 1  attack  to  represent  the  flow  of  their  team  cognition  in  response  to  the 
situation.  This  figure  (2a-b)  represents  the  macrocognitive  processes  employed  by  the  FDNY 
team  during  the  first  16  minutes  of  the  transcript  which  includes  209  speech  turns:  This  equals  a 
mean  of  13  speech  turns/minute,  indicating  the  rate  of  communications  was  fairly  dense.  We  see 
that  the  collaborative  process  was  highly  iterative  in  that  the  team  moved  back  and  forth  many 
times  between  the  knowledge  construction  stage  and  the  other  stages,  and  also  initiated  several 
actions  (courses  of  action/  requests  to  take  action)  as  depicted  in  the  last  column:  decision  to  take 
action. 


Phase  1 :  First  Plane  Hits  the  World  Trade  Center  (8:47  -  9:02) 


Knowledge  Team  Team  Outcome  Eval. 

Construction  Problem  Solving  Consensus  &  Revision 

Decision  to 
Take  Action 

8:48 

tk3 

C  TK>) 

8:49 

8:50 

DTI - r 

8:52 

* -  CMM-j  | 

8:53 

6««  KIO-, 

8:55 

ITK3 

<•  TK, 

C  IMMj, 

r  TK 

10IJ 

Phase  1 :  First  Plane  Hits  the  World  Trade  Center  (8:47  -  9:02am) 


Knowledge  Team  Team 

Construction  Problem  Solving  Consensus 

8:56 

Outcome  Eval. 

&  Revision 

Decision  to 
Take  Action 

■ORTA 

8:59  CMM-j 

9:00  F  TK - -5 

9K)1  - — -  CMMj 

9:02  ^  TSU 

ITKj 

^mTitk 

EBB 

Figure  2a-b.  Pattern  of  Team  Collaboration  Used  by  Firefighters  on  Sept.  11. 

The  lines  represent  the  order  in  which  the  team  moved  through  the  macrocognitive  stages 
included  in  the  model.  For  example,  in  Figure  2a,  the  first  speech  turn  involved  the  initial  state¬ 
ment  that  “We  just  had  a  plane  crash  into  upper  floors  of  the  World  Trade  Center.”  {team 
knowledge,  tk )  In  this  same  speech  turn,  this  firefighter  issued  the  course  of  action  to  “Transmit  a 
2nd  alarm  and  start  relocating  companies  into  the  area.”  (Fire  Department  of  the  City  of  New 
York,  2001).  This  initial  sizing  up  of  the  situation  resulted  in  the  immediate  decision  to  do 
something  about  the  situation  (e.g.,  issue  a  coa  to  bring  more  firefighting  assets  to  the  scene). 

The  goal  for  developing  this  set  of  figures  was  to  provide  a  sense  of  how  the  team  progressed 
through  the  phases  included  in  the  model  as  they  dealt  with  their  task.  Two  new  coding 
categories  are  included  in  these  figures  (these  new  categories  were  added  as  a  result  of  the 
analysis  conducted  for  the  three  tasks  reported  here).  Issuing  a  course  of  action  {coa)  and  request 
take  action  {rta)  are  the  by-product  of  the  previous  cognitive  processing  that  occurred  up  to  the 
decision  to  issue  the  coa/rta.  Therefore,  coa  and  rta  are  represented  in  a  fifth  column  -  outside 
the  four  columns  that  represent  the  four  stages  included  in  the  original  model  of  team 
collaboration. 

At  08:47  Battalion  1  called  the  Manhattan  dispatcher  to  report  the  crash  {team  knowledge 
development,  tk)  and  to  request  a  second  alarm,  providing  more  units  at  the  scene  {course  of 
action,  coa).  As  more  units  called  in  to  report  the  crash  (various  tk’s),  the  dispatcher  was  able  to 
form  a  mental  model  of  what  had  occurred.  Relaying  knowledge  back  to  the  dispatcher  is  critical 
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as  they  are  not  located  at  the  scene  of  the  attack;  anything  the  dispatcher  knows  comes  from  field 
reports  from  the  firefighters. 

Once  on  site  it  became  clear  to  the  firefighters  that  the  crash  might  have  been  an  intentional 
terror  attack  ( individual  mental  model  construction,  (irnni),  and  this  caused  the  team’s  mental 
model  to  change  ( convergence  of  individual  mental  model  ( cmm )  to  a  team  mental  model,).  In  a 
matter  of  minutes  the  FDNY  moved  through  all  four  stages  of  the  model  of  team  collaboration: 
knowledge  construction  and  collaborative  team  problem  solving  into  team  consensus  and  taking 
action  as  the  team  of  firefighters  agreed  that  more  units  were  needed  to  save  the  many  people 
trapped  inside  the  buildings.  Their  speed  of  collecting  information  and  making  decisions  saved 
countless  lives  in  various  buildings  at  the  WTC  Complex. 


Phase  2:  Second  Plane  Hits  the  World  Trade  Center  (9:03  -  9:58am) 
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Figure  3a-d.  Collaboration  Pattern  of  Firefighters  (2nd  Plane  hits  North  Tower). 

Moving  into  taking  action,  they  called  for  third  and  fourth  alarms,  revising  their  initial  plan. 
Furthermore,  when  the  second  plane  hit  the  South  Tower,  they  had  to  further  re-evaluate  their 
solution,  calling  for  a  fifth  alarm,  and  divide  their  resources  between  the  two  tower  lobbies  for 
command  posts.  Looking  at  the  attack  on  the  WTC  on  Sept.  1 1  as  one  large  problem,  the  FDNY 
never  reached  the  ultimate  outcome  evaluation  and  revision  stage.  Their  mental  model  did  not 
include  the  possibility  that  the  buildings  would  completely  collapse.  The  firefighters  were  under 
the  impression  that  they  were  working  within  a  much  larger  time  frame. 
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Figures  3a-3d  show  a  sample  of  the  team  communications  during  phase  two  of  the  attack  on  the 
WTC.  For  example,  the  communications  for  phase  two  began  with  the  statement  ‘You  have  a 
second  plane  into  the  other  tower  of  the  World  Trade  Center,  major  fire.’  (#1,  in  Figure  3  {tk). 
Table  3  includes  the  set  of  team  communications  selected  for  representation  in  the  figure  3a-d 
that  depict  the  team  collaboration  pattern  for  the  FDNY  during  phase  two,  when  the  second  plane 
hit  the  WTC.  (These  communications  are  a  subset  of  the  overall  communications  that  that 
transpired  during  this  phase  (9:03-9:58)  and  were  selected  to  represent  the  major  topics  discussed 
and  actions  taken  during  this  time  period.) 

Table  3.  Communications  Selected  to  Represent  Firefighters’  Pattern  of  Team 
Collaboration  During  Phase  2  of  the  Attack  on  the  World  Trade  Center.1 


9:02am  United  flight  175  flies  into  the  South  Tower  of  the  WTC 

Turn 

# 

Time 

Team  Communication 

Code 

1. 

9:03 

F.  You  have  a  second  plane  into  the  other  tower  of  the  Trade  Center,  major  fire. 

tk 

2. 

9:03 

Be  advised  on  the  83rd  floor,  room  8311,  we  have  people  trapped,  room  8311, 

83rd  floor.  Car  4  David  acknowledge. 

tk 

3. 

9:04 

F.  I'm  on  the  F.D.R.  Drive.  Definitely  something  hit  the  second  tower,  possibly 
two-thirds  of  the  way  up.  You've  got  visible  fire  showing  out  there. 

cmm 

4. 

9:05 

Fie  would  recommend  you  transmit  a  fifth  alarm  for  that  tower  as  well. 

cs 

5. 

9:06 

Best  to  have  the  [M.C.C.?]  and  have  our  personnel  be  secure  here  at  this  location. 

sa 

6. 

9;06 

D.  All  right.  Engine  1-4  remain  in  service  at  this  time.  Standby. 

coa 

7. 

9:06 

F.  All  right,  10-4.  Division  3  to  Manhattan,  call  leader  Car  4  David  on  the  scene, 

coa 

8. 

9:07 

F.  Did  you  give  me  the  box  that  I'm  being  assigned  to,  K? 

itk 

9. 

9:08 

All  units  responding  into  No.  1  World  Trade  Center  and  No.  2  World  Trade  Center, 
bring  all  additional  S.D.B.A.[?]  bottles  to  the  front  of  the  building.  All  units  to  box 

8087  and  998,  No.  1  and  No.  2  World  Trade  Center,  bring  your  extra  S.C.B.A. 
bottles  to  the  front  of  the  building,  as  per  the  division. 

coa 

10. 

9:09 

F.  ...  tower  would  you  like  us  to  be  starting  into,  Tower  1  or  Tower  2,  K? 

itk 

11. 

9:10 

D.  You're  going  to  2  World  Trade  Center,  K,  two. 

itk 

12. 

9:10 

F.  I  can't  pick  up  the  five  units  that  you  assigned  to  my  system  on.  Have  you  got  it 
in  the  computer  yet,  K? 

itk 

13. 

9:11 

F.  ...  1-0,  inform  everyone  assigned  to  the  scene  responding  on  West  Street  or  Liberty 

Street  not  to  pull  up  in  front  of  the  building.  We  have  ambulances  and  everybody  else 
pulling  up  and  we've  got  debris  falling  from  the  building.  They  have  to  stop  short  of  the 
building  either  north  or  south. 

cmm 

14. 

9:13 

F.  Would  you  advise  the  mobile  command  vehicle  to  come  in  on  West  and 

Liberty  Street,  West  and  Liberty  Street,  K. 

coa 

15. 

9:14 

D.  You  want  them  over  on  the  West  Side? 

itk 

16. 

9:16 

D.  Car  3  are  you  84  the  box? 

F.  We  are  10-84  the  box.  We  are  at  West  and  Vessey,  K. 

itk 

17. 

9:16 

D.  All  right.  Listen  1  have  some  floors  for  you  to  check  out 

coa 

18. 

9:16 

Car  3,  in  building  two,  the  No.  8-2  floor,  the  No.  8-8  floor  and  No.  8-9  floor. 

On  the  82nd  floor  it's  the  west.  I  have  other  floors. 

tk 

19. 

\ 

D.  O.K.,  the  83rd  floor  in  building  one;  the  104th  floor;  the  103rd  floor,  northwest  corner, 
room  103;  106th  floor;  83rd  floor  is  8-3-1-1  room;  and  the  82nd  floor,  east  side,  in  building 
one. 

tk 

20. 

F.  We're  the  marine  division,  we  will  position  by  the  Brooklyn  Bridge  for  a 
possible  transport  of  men  and  equipment  to  Manhattan. 

tk 

21. 

9:19 

F.  Have  Field  Comm,  report  in  front  of  the  Financial  District  Building  on 

West  Street ...  American  Express  immediately. 

coa 
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22. 

9:19 

F.  Give  me  the  company  identifications  that  are  coming  to  2  World  Trade 

Center.  Just  read  them  down. 

D.  All  right,  10-4:  Engine  2-1-1,  Ladder  [interference],  Engine  2-2,  Engine  5-3,  Engine  4- 
0,  Division  3,  Battalion  1-0,  Battalion  1-2,  Ladder  1-6,  Ladder  2,  Ladder  1-3,  Engine  2-2-1, 
Engine  2-3,  Engine  2-0-9,  Engine  2-1-2,  Engine  2-7-9,  Engine  2-3-0,  Engine  2-2-9,  Engine 
2-3-5,  Engine  2-2-0,  Engine  2-1-6,  Engine  2-1-7,  Engine  2-3-8,  Engine  2-1-4,  Ladder  12, 
Ladder  1-1-8,  Ladder  7,  Ladder  2-4,  High  Rise  1  and  Battalion  1-1,  Engine  7-4,  Engine  7- 
6,  Engine  4-7,  Engine  5-8,  Engine  9-1,  Ladder  2-2,  Ladder  2-5,  Ladder  3-5,  Four  Truck 
and  Ladder  2-1. 

itk 

23. 

9:22 

D.  All  right.  Field  Comm.  No.  1  World  Trade  Center,  the  1-0-3  floor,  southwest  comer  and 
northwest  corner,  reported  to  be  100  people  overcome  at  that  location.  Repeating,  No.  1 
World  Trade  Center,  103rd  floor,  northwest  [interference]  corner,  reported  to  be  100 
people  in  that  location.  Also,  Ladder  3  is  reporting  on  the  35th  floor  going  up  on  the 
stairwell  they've  got  numerous  injuries,  treating  numerous  injuries  from  burns  occupied  in 
the  stairwell  at  this  time.  Field  Comm,  receive. 

ct 

24. 

9:24 

F.  Brooklyn  Dispatch.  Urgent,  people  trapped,  5  World  Trade  on  the  8-0  floor;  3  World 
Trade,  that's  the  1-0-1  floor  and  the  1-0-2  floor.  Manhattan  receive? 

cu 

25. 

9:27 

F.  Flave  M.S.U.  activate  all  their  spares  and  bring  all  their  spares  and  all  spare  bottles  to  the 
scene  of  the  fifth  alarm,  No.  1  World  Trade  Center,  K. 

coa 

26. 

9:27 

D.  Field  Comm.,  No.  2  World  Trade  Center  on  the  8-3  and  the  8-4  floors  and  the  8-2  floor, 
people  trapped  at  this  time. 

cu 

27. 

9:30 

F.  Contact  the  units,  fifth  alarm,  coming  down  for  No.  2  World  Trade  Center,  contact  them 
individually  and  get  them  to  acknowledge  the  fact  that  they  are  to  come  to  Liberty  and 
West,  Liberty  and  West,  K. 

coa 

9:30 

F.  Marine  6  to  Manhattan,  in  the  event  of  a  transport  problem  into  Manhattan  we  can 
establish  a  staging  area  at  our  quarters. 

sa 

D.  Is  that  going  to  be  for  No.  2  or  No.  1  World  Trade. 

F.  Wasn't  sure.  I'd  say  go  with  both. 

(Fire  Department  of  the  City  of  New  York,  2001.) 


The  focus  of  the  collaboration  is  to  keep  all  team  members  apprised  of  the  situation  so  that  all 
team  members  can  maintain  overall  situation  awareness,  especially  when  dealing  with  as  large  a 
problem  situation  as  the  attack  on  the  WTC.  As  was  the  case  with  air  warfare  and  MIO  teams, 
the  9-11  firefighters  collaborated  more  about  the  ‘front  end’  of  the  problem:  ‘What’s  going  on?’ 
A  much  smaller  percentage  of  communications  were  devoted  to  collaboration  during  the  team 
consensus  and  outcome  evaluation  and  revision  phases  of  the  model.  For  both  air  warfare  and  the 
firefighters  many  decisions  were  made  all  along  the  way  of  dealing  with  the  problem  as  opposed 
to  making  one  big  final  decision  (course  of  actions)  and  reaching  a  consensus  on  that  course  of 
action.  If  this  same  pattern  holds  for  other  examples  of  team  collaboration  it  has  implications  for 
designing  collaboration  systems. 

The  iterative  pattern  of  moving  back  and  forth  between  the  phases  of  the  model  continues  during 
the  second  phase  of  the  FDNY  communications,  depicted  in  Figure  3.  The  team  moves  between 
knowledge  construction  and  team  problem  solving  and  many  times  issues  a  course  of  action  and 
then  moves  back  to  building  additional  team  knowledge.  In  some  cases  the  team  communications 
move  into  the  team  problem  solving  and  team  consensus  stages,  but  the  majority  of 
communications  entail  knowledge  construction  and  taking  action. 

The  patterns  included  in  Figures  2a-b  and  3a-d  depict  the  highly  iterative  process  used  by  the 
FDNY.  Each  time  they  noticed  something  new  they  immediately  issued  a  course  of  action  to  deal 
with  their  new  understanding  of  the  situation.  This  pattern  reflects  the  finely-honed  SOP  that  has 
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evolved  over  many  years  of  use  and  refinement.  Firefighters  are  trained  to  ‘size  up’  the  situation 
and  then  report  what  they  are  doing  about  it.  Information  gathering  was  an  on-going  process 
throughout  the  event. 


Phase  3:  The  South  Tower  Collapsed  (9:58-  10:27am) 
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Figure  4a-b.  Collaboration  Pattern  of  Firefighters  (The  South  Tower  Collapsed). 


As  seen  in  both  Figures  2a-b  and  3a-d,  the  team’s  collaboration  involved  a  highly  iterative 
process.  The  pattern  shows  that  each  time  they  noticed  something  new,  that  is,  when  the 
fire/situation  had  progressed  to  another  stage,  they  often  immediately  issued  a  course  of  action  to 
deal  with  their  new  understanding  of  the  situation.  For  example  the  first  utterance  in  Phase  1, 
shown  in  Figure  2,  was  to  inform  other  team  members  that  “We  just  had  a  plane  crash  into  upper 
floors  of  the  WTC.”  ( team  knowledge,  tk )  In  this  same  speech  turn,  this  firefighter  issues  the 
course  of  action  to  “Transmit  a  2nd  alarm  and  start  relocating  companies  into  the  area.”  ( coa )  It  is 
important  to  note  that  in  the  first  speech  turn  they  were  taking  action.  During  speech  turns  3-5 
(Figure  2)  {tk)  firefighters  are  describing  the  situation  to  others,  the  last  one  was  “we  have  a 
number  of  floors  on  fire.”  {tk)  In  speech  turn  #6  a  team  member  issued  another  coa :  “Transmit  a 
third  alarm  throughout  the  staging  area  at  Vessey  and  West  St.  As  the  third  alarm  assignment 
goes  into  that  area,  the  second  alarm  assignment  report  to  the  bldg.”  Speech  turn  #7  was: 
“...looked  like  it  was  intentional.  Inform  all  units  coming  in  from  the  back  it  could  be  a  terror 
attack.”  #8  followed:  “Roll  every  available  ambulance  you’ve  got  to  that  position.”  {coa) 


Phase  4:  The  North  Tower  Collapsed  (1 0:28  - 1 1 :07am) 
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Phase  4:  The  North  Tower  Collapsed  (1 0:28  - 1 1 :07am) 
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Figure  5a-c.  Collaboration  Pattern  of  Firefighters  (The  North  Tower  Collapsed). 

The  team  kept  moving  back  to  the  knowledge  construction  stage  the  entire  time  because  they 
were  constantly  passing  information  and  making  mental  model  adjustments  to  keep  each  other 
and  the  dispatcher  infonned  of  changes  at  the  scene.  When  the  South  tower  collapsed  the  mission 
in  the  North  tower  became  one  of  strict  evacuation.  Most  of  the  collaborative  team  problem 
solving  was  done  at  the  Incident  Command  posts,  where  the  Chiefs  collaborated  to  create  a  plan 
while  they  filtered  information  for  currency  and  factualness. 

Team  Consensus.  FDNY  Chiefs  at  the  scene  were  able  to  decide  on  solution  options  quickly 
because  the  FDNY  SOP  for  regular  response  options  was  applicable  for  many  instances. 
However,  dealing  with  an  attack  on  scale  of  Sept  1 1  meant  there  were  cases  were  SOPs  were  not 
available.  Fire  chiefs  moved  between  collaborative  team  problem  solving  and  team  consensus 
stages  regarding  solution  options  for  various  problems. 

Outcome  Evaluation  and  Revision.  For  smaller,  more  manageable  problems,  the  FDNY  was 
able  to  reach  a  consensus  and  go  through  entire  collaborative  cycle.  For  example,  getting 
firefighters  down  via  elevators  usually  works,  but  plans  were  modified  to  use  stairs  to  climb  to 
the  fire.  For  the  larger  problem  of  evacuating  civilians  and  putting  out  the  fire,  this  final  goal  was 
never  reached  because  buildings  collapsed  before  they  had  time  to  see  their  solution  through  to 
completion. 
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Figure  6a-c.  Collaboration  Pattern  for  Air  Warfare  Team. 

A  different  pattern  of  collaboration  is  evident  in  figure  6a-c,  which  depicts  a  high-level 
representation  of  the  macrocognitive  processes  employed  by  the  air  warfare  team.  The  vast 
majority  of  their  communications  involved  macrocognitive  processes  during  the  knowledge 
construction  phase  as  well  as  several  decisions  to  take  action. 
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Figure  7a-c.  Collaboration  Pattern  for  Maritime  Interdiction  Operations  Team. 
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As  shown  in  Figure  7a-c,  the  MIO  team  engaged  in  more  team  problem  solving  in  addition  to 
knowledge  construction  and  decision  to  take  action. 

New  Coding  Categories:  Decision  to  Taking  Action 

Many  critical  tasks  that  involve  team  collaboration  include  team  members  taking  action  in 
addition  to  developing  situation  awareness  and  agreeing  on  a  final  COA.  Various  actions  are 
taken  as  part  of  the  overall  information  gathering  process  (e.g.,  MIOs,  air  warfare,  firefighters, 
etc.).  Problem-solving  tasks  also  involve  taking  actions  as  part  of  the  overall  decision  making 
and  COA  implementation.  Many  tasks  involve  a  series  of  decision  making  and  action  taking  in 
order  to  accomplish  the  mission.  For  example,  in  MIOs  members  of  the  boarding  party 
physically  search  the  ship  using  sensing  equipment  to  take  various  readings  which  are  sent  to 
experts  at  reachback  centers  for  analysis.  In  some  cases,  members  of  the  boarding  party  are  then 
asked  to  take  additional  readings  to  provide  more  fine-grained  data  that  will  help  more  precisely 
determine  the  type  of  cargo  on  the  ship.  Similarly,  biometric  data  and  video  data  is  collected  and 
sent  back  for  analysis.  This  process  of  physically  searching  the  ship  for  contraband  cargo  and 
suspect  people  entails  a  series  of  actions,  thus,  many  speech  turns  involve  requesting  a  team 
member  to  take  an  action.  These  requests  can  involve  one  team  member  to  another  (peer-to-peer 
situation)  or  a  superior  talking  to  a  subordinate. 

Two  new  coding  categories  were  added  to  the  set  of  macrocognitive  process  definitions  that  are 
included  in  the  model.  The  first  involves  a  person  in  command  issuing  an  order  regarding  a 
course  of  action,  that  is,  a  person  with  higher  rank  tells  them  to  take  some  specific  action  against 
a  potential  threat  track.  This  could  include  issuing  verbal  warnings,  illuminating  or  locking-on 
with  radar,  developing  a  firing  solution,  covering  with  missiles,  etc.  Several  examples  for  both  of 
these  new  coding  categories  are  listed  in  Tables  4  and  5  to  illustrate  the  qualitative  difference 
between  issuing  and  order  doe  a  COA  and  request  take  action  -  where  one  peer  asks  another  peer 
to  do  something. 

Distinctions  between  these  two  new  categories  that  involve  a  decision  to  take  action  are  the  (1) 
relationship  between  the  people  and  (2)  the  urgency  the  action  requires.  Issuing  an  order 
regarding  a  course  of  action  involves  a  person  with  higher  rank  (e.g.,  commanding  officer 
speaking  to  the  tactical  action  officer  (TAO),  or  the  TAO  speaking  to  one  of  the  enlisted  system 
operators),  telling  a  subordinate  to  take  some  specific  action  against  a  potential  threat  track. 
Table  4  lists  examples  of  courses  of  action  ( coa )  found  in  the  three  decision-making  domains 
analyzed.  These  actions  might  include  issuing  verbal  warnings,  illuminating  or  locking-on  with 
radar,  developing  a  firing  solution,  covering  with  missiles,  etc.  Request  a  team  member  take 
action  refers  to  telling  a  team  member  to  do  something  but  it  is  not  a  direct  action  against  a  threat 
track.  For  example,  “Can  you  try  and  change  7006  and  7005  to  assumed  hostile?  I  keep  trying 
and  can’t  get  it  to  do  it.” 


Table  4.  Examples  of  Course  of  Action  (coa)  Macrocognitive  Process. 


9-11  Firefighters 

Transmit  a  second  alarm  and  start  relocating  companies  into  the  area. 

Send  every  available  ambulance,  everything  you’ve  got,  to  the  WTC. 

All  units  respond  into  West  St.  Transmit  a  10-60  also. 

Get  us  a  staging  area. .  .somewhere  on  West  St. 

I  want  all  but  one  of  them  here. 

Division  to  Manhattan  calling  Car  4  David  K. 

Units  calling  Manhattan,  one  at  a  time. 
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All  incoming  units  into  World  1  and  World  2  are  to  bring  additional  cylinders. 

MIO 

Search  team  will  report  size  of  material  and  its  current  containment 
condition;  then  make  recommendations. 

But,  if  you  have  plutonium,  you  need  to  confiscate. 

Air  Warfare 

Cover  tracks  7005  and  7006  with  birds 

Cover  8032  (TN  7013)  with  standard  missile  also  generate  a  SWG  1A  solution. 

The  second  new  coding  category  is  request  team  member  take  an  action,  which  involves  asking  a 
team  member  to  do  something.  Table  5  lists  examples  of  request  take  action  ( tra )  found  in  the 
three  decision-making  domains  analyzed.  The  difference  between  these  two  categories  is  that  the 
later  does  not  involve  a  direct  action  against  a  threat  track,  e.g.,  ‘Can  you  try  and  change  7006 
and  7005  to  assumed  hostile?’  Much  of  the  radio  communications  for  the  firefighters  also 
involves  asking  fellow  firefighters  to  respond  or  do  something  towards  the  team’s  goal.  Request 
take  action  involves  asking  a  team  member  to  take  an  action  that  will  help  move  the  problem 
along  in  terms  of  accomplishing  the  overall  mission,  e.g,  ‘Would  you  relocate  the  only  rescue 
that’s  not  going  to  the  Trade  Center,  put  them  in  Rescue  1  in  Manhattan,  please,’  or:  ‘Will  you 
call  our  bosses  downtown  and  have  them  secure  the  MCC?’  Rta  is  generally  a  type  of  action  that 
has  less  urgency  associated  with  it. 

A  coa  involves  a  superior  telling  a  subordinate  to  do  something  and  the  nature  of  the  action  is 
more  severe  (e.g.,  firing  a  missile)  RTA  is  usually  peer-to-peer  request  and  the  action  requested 
is  of  a  lesser  consequence.  Differences  between  a  coa  and  rta  center  on  the  degree  of  authority 
held  by  the  speaker  and  the  urgency  with  which  the  action  needs  to  be  executed. 


Table  5.  Examples  of  Request  Take  Action  (rta)  Macrocognitive  Process. 


9-11  Firefighters 

We’re  going  to  need  the  P.D.  for  security  on  the  entire  World  Trade  Center. 

Please  have  ambulances  respond  to  West  Street. 

Would  you  relocate  the  only  rescue  that’s  not  going  to  the  Trade  Center, 
put  them  in  Rescue  1  in  Manhattan,  please. 

Will  you  call  our  bosses  downtown  and  have  them  secure  the  MCC? 

MIO 

Mark  material  for  confiscation. 

Recommend  material  be  confiscated. 

Make  sure  you  handle  carefully. 

Unless  positive  that  destination  is  country  under  full-scope  IAEA 

Safeguards,  need  to  confiscate. 

Given  multiple  radiation  hits  and  suspect  equipment  RECOMMEND  Divert  of 
entire  ship  to  Friendly  Port  (If  we  are  u/w)  or  detained  and  moved  to  a  safe 
location  (if  in  US) 

Air  Warfare 

Let’s  investigate  with  (combat  air  patrol)  CAP. 

Confirm  that  tracks  originating  from  Iranian  air  space  are  designated  unknown 
assumed  hostile. 

Go  ahead  and  tag  8037  and  company  assumed  hostile. 

DISCUSSION 

Several  trends  are  evident  when  examining  the  results  depicted  in  Figures  2-7.  First,  all  teams 
spent  the  majority  of  their  time  in  the  knowledge  construction  phase.  The  attack  on  the  WTC  was 
a  very  unique  situation  that  did  not  resemble  anything  the  FDNY  had  previously  experienced. 
This  high  level  of  novelty  more  than  likely  contributed  to  the  team  needing  to  devote  an 
inordinate  amount  of  time  to  constructing  their  mental  model  of  the  situation. 
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The  second  noteworthy  point  is  that  for  both  the  firefighters  on  Sept.  11,  and  the  air  warfare  and 
MIO  teams,  the  overall  collaboration  process  did  not  culminate  in  one  final  outcome.  While  in 
other  firefighting  scenarios  the  final  outcome  is  putting  out  the  fire,  the  team  does  not  necessarily 
collaborate  to  decide  on  the  final  outcome,  i.e.,  whether  one  option/  final  plan  is  better  than 
another.  (For  example,  the  way  a  team  of  representatives  from  several  cities  who  are 
collaborating  to  decide  on  the  best  final  plan  for  where  to  locate  a  trash  incinerator  or  a  nuclear 
storage  facility  would.)  A  third  important  point  is  that  for  the  kinds  of  tactical  decision-making 
problems  reported  here,  many  actions  are  taken  all  along  the  entire  process  of  dealing  with  the 
situation,  whether  it  involves  dealing  with  potential  air  threats,  a  MIO,  or  a  fire  -  and  all  these 
decisions  and  associated  coas  are  critical  to  dealing  with  the  situation.  These  may  be  considered 
mini-decisions  in  that  they  are  not  major  or  final  decisions,  however,  they  are  all  considered  to 
be  critical  in  terms  of  the  teams’  handling  the  problem  situation. 

CONCLUSIONS 

For  the  9-11  data,  individual  task  knowledge  development  (itk)  is  by  far  the  category  with  the 
greatest  percentage  of  codes  at  42%.  Firefighters  are  asking  a  great  deal  of  questions,  and  while 
asking  questions  is  encouraged  for  exchanging  team  knowledge,  it  begs  the  question  of  whether 
there  is  a  way  to  alleviate  some  of  the  questions  by  providing  more  information  to  the  responding 
units  and  in  providing  a  decision  support  tool.  Many  communications  dealt  with  locations  for 
staging  areas.  Incorporating  a  navigation  system  into  the  handie  talkies  could  help  a  lot  with  map 
issues.  This  also  applies  to  the  category  with  the  second  highest  number  of  codes,  team 
knowledge  development  (tk),  with  210  out  of  771  (27%). 

Very  few  of  the  cognitive  processes  included  in  the  team  consensus  stage  were  evident  in  the 
team  communications  that  transpired  during  these  three  collaborative  tasks.  All  three  tasks  are 
tactical  level  tasks;  that  is,  these  operations  are  conducted  within  a  very  short  time  period.  Due  to 
the  speed  of  a  potentially  hostile  aircraft,  political/  economic  pressure  to  conduct  the  MIO 
quickly,  and  for  obvious  reasons  for  the  firefighters,  these  decisions  need  to  be  made  within 
minutes,  or  even  seconds,  as  the  situation  unfolds.  In  contrast,  operational  and  strategic  level 
operations  occur  over  longer  time  intervals,  e.g.,  weeks  or  months.  More  of  the  macrocognitive 
processes  included  in  the  collaborative  team  problem  solving  and  team  consensus  stages  may 
occur  in  collaborative  problem-solving  tasks  that  span  a  longer  time  period. 

During  the  collaborative  team  problem  solving  stage  the  most  frequently  used  process  varied 
between  all  three  decision-making  domains.  The  macrocognitive  processes  with  the  highest  use 
for  each  decision-making  domain  were:  air  warfare-team  shared  understanding;  MIO-iterative 
collection  and  analysis;  firefighting-convergence  of  mental  models.  These  differences  reflect 
differences  in  the  tasks  performed  for  these  three  domains.  Air  warfare  teams  focus  on 
developing  team  shared  situation  awareness  regarding  the  status  (threat/non-threat)  of  the  various 
tracks  (air  and  surface)  in  the  vicinity  of  the  ship.  MIO  teams  focus  on  iterative  collection  of 
data,  and  receiving  feedback  on  results  of  analysis  of  this  data  which  drives  additional  data 
collection  until  they  are  certain  regarding  the  types  of  materials  found  on  the  vessel.  For  the  9-11 
firefighting  task,  many  completely  unanticipated  events  occurred  which  made  it  necessary  for  the 
firefighters  to  develop  a  new  mental  model  of  the  situation  they  were  confronted  with  in 
response  to  each  new  event:  a  terrorist  attack,  the  collapse  of  WTC  1,  collapse  of  WTC  2,  and 
not  being  able  to  rescue  people  due  to  the  collapse  of  the  two  buildings. 
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